Empowerment for Continuous Agent-Environment Systems

This paper develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowemerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, e.g., it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this paper is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte-Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning

Local information transfer as a spatiotemporal filter for complex systems

We present a measure of local information transfer, derived from an existing averaged information-theoretical measure, namely transfer entropy. Local transfer entropy is used to produce profiles of the information transfer into each spatiotemporal point in a complex system. These spatiotemporal profiles are useful not only as an analytical tool, but also allow explicit investigation of different parameter settings and forms of the transfer entropy metric itself. As an example, local transfer entropy is applied to cellular automata, where it is demonstrated to be a novel method of filtering for coherent structure. More importantly, local transfer entropy provides the first quantitative evidence for the long-held conjecture that the emergent traveling coherent structures known as particles (both gliders and domain walls, which have analogues in many physical processes) are the dominant information transfer agents in cellular automata.Comment: 12 page

A framework for the local information dynamics of distributed computation in complex systems

The nature of distributed computation has often been described in terms of the component operations of universal computation: information storage, transfer and modification. We review the first complete framework that quantifies each of these individual information dynamics on a local scale within a system, and describes the manner in which they interact to create non-trivial computation where "the whole is greater than the sum of the parts". We describe the application of the framework to cellular automata, a simple yet powerful model of distributed computation. This is an important application, because the framework is the first to provide quantitative evidence for several important conjectures about distributed computation in cellular automata: that blinkers embody information storage, particles are information transfer agents, and particle collisions are information modification events. The framework is also shown to contrast the computations conducted by several well-known cellular automata, highlighting the importance of information coherence in complex computation. The results reviewed here provide important quantitative insights into the fundamental nature of distributed computation and the dynamics of complex systems, as well as impetus for the framework to be applied to the analysis and design of other systems.Comment: 44 pages, 8 figure

Helicobacter pylori's historical journey through Siberia and the Americas.

The gastric bacterium Helicobacter pylori shares a coevolutionary history with humans that predates the out-of-Africa diaspora, and the geographical specificities of H. pylori populations reflect multiple well-known human migrations. We extensively sampled H. pylori from 16 ethnically diverse human populations across Siberia to help resolve whether ancient northern Eurasian populations persisted at high latitudes through the last glacial maximum and the relationships between present-day Siberians and Native Americans. A total of 556 strains were cultivated and genotyped by multilocus sequence typing, and 54 representative draft genomes were sequenced. The genetic diversity across Eurasia and the Americas was structured into three populations: hpAsia2, hpEastAsia, and hpNorthAsia. hpNorthAsia is closely related to the subpopulation hspIndigenousAmericas from Native Americans. Siberian bacteria were structured into five other subpopulations, two of which evolved through a divergence from hpAsia2 and hpNorthAsia, while three originated though Holocene admixture. The presence of both anciently diverged and recently admixed strains across Siberia support both Pleistocene persistence and Holocene recolonization. We also show that hspIndigenousAmericas is endemic in human populations across northern Eurasia. The evolutionary history of hspIndigenousAmericas was reconstructed using approximate Bayesian computation, which showed that it colonized the New World in a single migration event associated with a severe demographic bottleneck followed by low levels of recent admixture across the Bering Strait

Evidence for surprise minimization over value maximization in choice behavior

Classical economic models are predicated on the idea that the ultimate aim of choice is to maximize utility or reward. In contrast, an alternative perspective highlights the fact that adaptive behavior requires agents' to model their environment and minimize surprise about the states they frequent. We propose that choice behavior can be more accurately accounted for by surprise minimization compared to reward or utility maximization alone. Minimizing surprise makes a prediction at variance with expected utility models; namely, that in addition to attaining valuable states, agents attempt to maximize the entropy over outcomes and thus 'keep their options open'. We tested this prediction using a simple binary choice paradigm and show that human decision-making is better explained by surprise minimization compared to utility maximization. Furthermore, we replicated this entropy-seeking behavior in a control task with no explicit utilities. These findings highlight a limitation of purely economic motivations in explaining choice behavior and instead emphasize the importance of belief-based motivations

Integrated information increases with fitness in the evolution of animats

One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular since organismal functional complexity is not well-defined. We present here several candidate measures that quantify information and integration, and study their dependence on fitness as an artificial agent ("animat") evolves over thousands of generations to solve a navigation task in a simple, simulated environment. We compare the ability of these measures to predict high fitness with more conventional information-theoretic processing measures. As the animat adapts by increasing its "fit" to the world, information integration and processing increase commensurately along the evolutionary line of descent. We suggest that the correlation of fitness with information integration and with processing measures implies that high fitness requires both information processing as well as integration, but that information integration may be a better measure when the task requires memory. A correlation of measures of information integration (but also information processing) and fitness strongly suggests that these measures reflect the functional complexity of the animat, and that such measures can be used to quantify functional complexity even in the absence of fitness data.Comment: 27 pages, 8 figures, one supplementary figure. Three supplementary video files available on request. Version commensurate with published text in PLoS Comput. Bio

Peripheral Administration of a Humanized Anti-PrP Antibody Blocks Alzheimer's Disease Aβ Synaptotoxicity.

Alzheimer's disease (AD) is associated with pathological assembly states of amyloid-β protein (Aβ). Aβ-related synaptotoxicity can be blocked by anti-prion protein (PrP) antibodies, potentially allowing therapeutic targeting of this aspect of AD neuropathogenesis. Here, we show that intravascular administration of a high-affinity humanized anti-PrP antibody to rats can prevent the plasticity-disrupting effects induced by exposure to soluble AD brain extract. These results provide an in vivo proof of principle for such a therapeutic strategy

Sensitive detection of Aβ protofibrils by proximity ligation - relevance for Alzheimer's disease

<p>Abstract</p> <p>Background</p> <p>Protein aggregation plays important roles in several neurodegenerative disorders. For instance, insoluble aggregates of phosphorylated tau and of Aβ peptides are cornerstones in the pathology of Alzheimer's disease. Soluble protein aggregates are therefore potential diagnostic and prognostic biomarkers for their cognate disorders. Detection of the aggregated species requires sensitive tools that efficiently discriminate them from monomers of the same proteins. Here we have established a proximity ligation assay (PLA) for specific and sensitive detection of Aβ protofibrils via simultaneous recognition of three identical determinants present in the aggregates. PLA is a versatile technology in which the requirement for multiple target recognitions is combined with the ability to translate signals from detected target molecules to amplifiable DNA strands, providing very high specificity and sensitivity.</p> <p>Results</p> <p>For specific detection of Aβ protofibrils we have used a monoclonal antibody, mAb158, selective for Aβ protofibrils in a modified PLA, where the same monoclonal antibody was used for the three classes of affinity reagents required in the assay. These reagents were used for detection of soluble Aβ aggregates in solid-phase reactions, allowing detection of just 0.1 pg/ml Aβ protofibrils, and with a dynamic range greater than six orders of magnitude. Compared to a sandwich ELISA setup of the same antibody the PLA increases the sensitivity of the Aβ protofibril detection by up to 25-fold. The assay was used to measure soluble Aβ aggregates in brain homogenates from mice transgenic for a human allele predisposing to Aβ aggregation.</p> <p>Conclusions</p> <p>The proximity ligation assay is a versatile analytical technology for proteins, which can provide highly sensitive and specific detection of Aβ aggregates - and by implication other protein aggregates of relevance in Alzheimer's disease and other neurodegenerative disorders.</p

Interaction between prion protein and toxic amyloid beta assemblies can be therapeutically targeted at multiple sites

A role for PrP in the toxic effect of oligomeric forms of A beta, implicated in Alzheimer's disease (AD), has been suggested but remains controversial. Here we show that PrP is required for the plasticity-impairing effects of ex vivo material from human AD brain and that standardized A beta-derived diffusible ligand (ADDL) preparations disrupt hippocampal synaptic plasticity in a PrP-dependent manner. We screened a panel of anti-PrP antibodies for their ability to disrupt the ADDL-PrP interaction. Antibodies directed to the principal PrP/A beta-binding site and to PrP helix-1, were able to block A beta binding to PrP suggesting that the toxic A beta species are of relatively high molecular mass and/or may bind multiple PrP molecules. Two representative and extensively characterized monoclonal antibodies directed to these regions, ICSM-35 and ICSM-18, were shown to block the A beta-mediated disruption of synaptic plasticity validating these antibodies as candidate therapeutics for AD either individually or in combination